[dockers] Prevent old supervisord messages from gettting re-logged to syslog #2439
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
what overhead are we adding here? could this slow down container stop or start for some time critical task such as warm reboot, fast-reboot? |
pavel-shirshov
approved these changes
Jan 17, 2019
|
@lguohan: The overhead should be negligible. Supervisor doesn't write very many log messages. However, we could take measurements with and without the change to be certain. |
|
Made to 201811 branch on 1/25/2019 |
yxieca
pushed a commit
that referenced
this pull request
Jan 25, 2019
theasianpianist
added a commit
to theasianpianist/sonic-buildimage
that referenced
this pull request
Sep 2, 2022
Introduce following commits: - eedf59d [intfmgr]: Enable `accept_untracked_na` kernel param (sonic-net#2439) - bbd4eb7 [BFD]Clean up state_db BFD entries on swss restart (sonic-net#2434) (sonic-net#2435) - aa7b546 [FDB] [202012] Fix fbdorch to properly handle syncd FDB FLUSH Notif (sonic-net#2401) - 4190c13 Support for platforms based on Clounix Networks' device (sonic-net#2399) Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
theasianpianist
added a commit
that referenced
this pull request
Sep 3, 2022
Include following new kernel commits: - 815cb34 [patch]: Add accept_untracked_na kernel param (#291) - fbb76cc Remove kernel config SERIAL_8250_RUNTIME_UARTS (#286) Introduce following SWSS commits: - eedf59d [intfmgr]: Enable `accept_untracked_na` kernel param (#2439) - bbd4eb7 [BFD]Clean up state_db BFD entries on swss restart (#2434) (#2435) - aa7b546 [FDB] [202012] Fix fbdorch to properly handle syncd FDB FLUSH Notif (#2401) - 4190c13 Support for platforms based on Clounix Networks' device (#2399) Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
dprital
added a commit
to dprital/sonic-buildimage
that referenced
this pull request
Jan 30, 2023
Update sonic-utilities submodule pointer to include the following: * 75d233f [system-health] Fix issue: show system-health CLI crashes ([sonic-net#2635](sonic-net/sonic-utilities#2635)) * 5782da4 Fixed admin state config CLI for Backport interfaces ([sonic-net#2557](sonic-net/sonic-utilities#2557)) * c4c6808 suppport multi asic for show queue counter ([sonic-net#2439](sonic-net/sonic-utilities#2439)) * 1b21201 [show_bfd] add local discriminator in show bfd command ([sonic-net#2625](sonic-net/sonic-utilities#2625)) Signed-off-by: dprital <drorp@nvidia.com>
8 tasks
liat-grozovik
pushed a commit
that referenced
this pull request
Feb 2, 2023
Update sonic-utilities submodule pointer to include the following: * 75d233f [system-health] Fix issue: show system-health CLI crashes ([#2635](sonic-net/sonic-utilities#2635)) * 5782da4 Fixed admin state config CLI for Backport interfaces ([#2557](sonic-net/sonic-utilities#2557)) * c4c6808 suppport multi asic for show queue counter ([#2439](sonic-net/sonic-utilities#2439)) * 1b21201 [show_bfd] add local discriminator in show bfd command ([#2625](sonic-net/sonic-utilities#2625)) Signed-off-by: dprital <drorp@nvidia.com>
8 tasks
StormLiangMS
added a commit
that referenced
this pull request
Feb 17, 2023
Why I did it Submodule advances: sonic-utilities 8e8e6088 - [202211][dhcp_relay] Remove add field of vlanid to DHCP_RELAY table while adding vlan ([201811 sub-module] advance sub-modules: utilities, swss, swss-common #2679) (16 hours ago) [Yaqiang Zhu] 1400fb94 - [GCU] Ignore bgpraw in GCU applier (Fix sfputil indexing for 7170-Q59S20 #2623) (15 hours ago) [jingwenxie] f76a6364 - [vlan] Refresh dhcpv6_relay config while adding/deleting a vlan ([sonic-py-swsssdk] Update submodule #2660) (15 hours ago) [Yaqiang Zhu] 7849e18d - [db_migrator] make LOG_LEVEL_DB migration more robust (Mellanox platform: attach queues 2 and 6 to lossy profile using generic buffer template #2651) (16 hours ago) [Stepan Blyshchak] c7df6dfa - Fixed a bug in "show vnet routes all" causing screen overrun. (Add hook to allow customizing link cable lengths #2644) (16 hours ago) [siqbal1986] a5505f02 - show logging CLI support for logs stored in tmpfs (Traceback error seen while issuing show interface commands with if_names #2641) (16 hours ago) [mihirpat1] bbacb91a - [system-health] Fix issue: show system-health CLI crashes (Updating deb package for platform and sai #2635) (16 hours ago) [Junchao-Mellanox] 8d724024 - [sai_failure_dump]Invoking dump during SAI failure ([dockers]: Upgrade LLDP docker to stretch build #2633) (16 hours ago) [Sudharsan Dhamal Gopalarathnam] 3c3be526 - Add transceiver info CLI support to show output from TRANSCEIVER_INFO for ZR ([submodule]: Update sonic-sairedis pointer #2630) (16 hours ago) [mihirpat1] 37f41666 - [show] add support for gRPC show commands for active-active ([bitmap-vnet]: Bitmap vnet test image [DO NOT MERGE] #2629) (16 hours ago) [vdahiya12] b06d7fe4 - [show_bfd] add local discriminator in show bfd command ([Pmon] Selectively load pmon container daemons #2625) (16 hours ago) [Baorong Liu] 6adcd3e8 - [GCU] Ignore bgpraw table in GCU operation ([Mellanox] Fix SAI version #2628) (16 hours ago) [jingwenxie] c65bdc35 - [muxcable][config] Add support to enable/disable ceasing to be an advertisement interface when radv service is stopped (Add knob in ConfigDB to enable/disable telemetry container #2622) (16 hours ago) [Jing Zhang] 91e9457f - Add Transceiver PM basic CLI support to show output from TRANSCEIVER_PM table for ZR ([201803] Restart SwSS, syncd and dependent services if a critical process in syncd container exits #2615) (16 hours ago) [longhuan-cisco] 54cc8c5a - Remove TODO comment which is no longer relevant (Warm-reboot: teamd warm restart caused neighbor deleted and learned again. #2600) (16 hours ago) [Lior Avramov] 6891b4fb - Making 'show feature autorestart' more resilient to missing auto_restart config in CONFIG_DB ([submodule] update mellanox hw-mgmgt pointer (V.2.0.0061) #2592) (16 hours ago) [kartik-arista] 1e8bea37 - [storyteller] add link prober state change to story teller ([sonic-buildimage] New feature managementVRF(L3mdev) #2585) (16 hours ago) [Jing Zhang] 7481a20f - Extend fast-reboot STATE_DB entry timer ([submodule]: update sonic-swss-common, sonic-py-swsssdk, sonic-snmpagent #2577) (16 hours ago) [Aryeh Feigin] 0e08701c - [sonic_installer] use /etc/resolv.conf from the host when migrating packages (Set a rate limit on syslog messages from all Docker containers #2573) (16 hours ago) [Stepan Blyshchak] 06096780 - Fixed admin state config CLI for Backport interfaces (Prior to install a new ONIE SONiC image, delete all partitions except EFI/ONIE #2557) (16 hours ago) [anamehra] 9f1f13e4 - [show] Add bgpraw to show run all (Fixed typo on paragraph #40 #2537) (16 hours ago) [jingwenxie] 98bc8bd2 - [chassis][voq] Add "show fabric reachability" command. ([ntp]: Build 4.2.6 locally. #2528) (16 hours ago) [jfeng-arista] 3a50b63f - Preserve copp tables through DB migration ([docker-radvd]: upgrade docker radvd to stretch based #2524) (16 hours ago) [Aryeh Feigin] 28f6b127 - [masic] 'show interfaces counters' reminds to use '-d all' option to check for internal links (solve dependency issue #2466) (16 hours ago) [wenyiz2021] 15026e14 - suppport multi asic for show queue counter ([dockers] Prevent old supervisord messages from gettting re-logged to syslog #2439) (16 hours ago) [zhixzhu] 2d773e17 - [masic support] 'show run bgp' support for multi-asic (lo address not synced to the asic #2427) (16 hours ago) [wenyiz2021] sonic-swss 4f304bc - [EVPN]Handling race condition when remote VNI arrives before tunnel map entry ([sonic-quagga] Function defect, do NOT cancel route while connect IP down #2642) (15 hours ago) [Sudharsan Dhamal Gopalarathnam] 34fc615 - [sai_failure_dump]Invoking dump during SAI failure (Add hook to allow customizing link cable lengths #2644) (15 hours ago) [Sudharsan Dhamal Gopalarathnam] b817695 - [autoneg]Fixing adv interface types to be set when AN is disabled (Fix issue with platform file path name #2638) (15 hours ago) [Sudharsan Dhamal Gopalarathnam] ab36bd4 - [bfdorch] add local discriminator to state DB ([bitmap-vnet]: Bitmap vnet test image [DO NOT MERGE] #2629) (15 hours ago) [Baorong Liu] 6343471 - Remove TODO comments that are no longer relevant (Add knob in ConfigDB to enable/disable telemetry container #2622) (15 hours ago) [Lior Avramov] 2b1869c - [refactor]Refactoring sai handle status (Rollback kernel submodule update. #2621) (15 hours ago) [Sudharsan Dhamal Gopalarathnam] c41a1b7 - Fix issue ARP entry is out of sync between kernel and APPL_DB after warm reboot if the ARP entry is updated more than once during warm reboot in PFC watchdog warm reboot test #13341 ARP entry can be out of sync between kernel and APPL_DB if multiple updates are received from RTNL ([sub module] advance sonic-utilities sub module for 201811 branch #2619) (15 hours ago) [Stephen Sun] da0cf7a - Changed the BFD default detect multiplier to 10x ("failed to load plugin io.containerd.snapshotter..." seen during linux boot up #2614) (15 hours ago) [siqbal1986] 13b5adf - [vstest] Only collect stdout of orchagent_restart_check in vstest ([submodules] update swss and utilities pointers #2597) (15 hours ago) [bingwang-ms] 2b9d94d - Avoid aborting orchagent when setting TUNNEL attributes (build failing for PLATFORM=p4 #2591) (15 hours ago) [Stephen Sun] 99b7d3b - Only collect stdout of orchagent_restart_check in vstest ( [saibcm-modules]: import new bcm modules #2578) (15 hours ago) [bingwang-ms] 5209c42 - dereg acl-rule counters during acl-table del ([201803] Set a rate limit on syslog messages from all Docker containers #2574) (15 hours ago) [Vivek] ae68054 - Fixed set mtu for deleted subintf due to late notification ([vs]: Add option to specify platform name for DVS orchagent #2571) (15 hours ago) [EdenGri] ab13dfa - Remove TODO comments which are no longer needed (support set timezone in ConfigDB #2568) (15 hours ago) [Junchao-Mellanox] a3545cf - Modify coppmgr mergeConfig to support preserving copp tables through reboot. (Added new SN3700/SN3700C Mellanox platforms #2548) (15 hours ago) [Aryeh Feigin] be16e79 - Use github code scanning instead of LGTM ([201803] [services] Restart SwSS service upon unexpected critical process exit #2546) (15 hours ago) [Liu Shilong] 63c0234 - Updated handling of VRF_VNI mapping and VLAN_VNI mapping for same VNI ID (Move warm_restart enable/disable config to stateDB WARM_RESTART_ENABL… #2538) (15 hours ago) [Tapash Das] 4844111 - Fix potential risks ([mlnx] Fix sai xml path for boxer platform #2516) (15 hours ago) [Liran-Ar] 6420808 - [p4orch]: PINS Extension tables support ([build] When generating image version, handle case where current commit has no reachable tags #2506) (15 hours ago) [svshah-intel] sonic-swss-common 1badd46 - Increase the netlink buffer size from 3MB to 16MB. (arp_update doesn't sleep 300 between each execution #739) (14 hours ago) [KISHORE KUNAL] 6555057 - Refactor eventpublisher deinit ([acl] Add default deny rule for l3 table #734) (14 hours ago) [Zain Budhwani] f4d6de7 - Use github code scanning instead of LGTM ([sonic-quagga]:update submodule #718) (14 hours ago) [Liu Shilong] sonic-linux-kernel 74f9a8f - Update linux kernel for hw-mgmt V.7.0020.4104 (Move template files to /usr/share/sonic/templates #305) (14 hours ago) [Stephen Sun] 6365701 - Fixes for emmc unreliability ([build_debian.sh]: Integrate system dump script #270) (14 hours ago) [Samuel Angebault] How I did it How to verify it
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Upon restarting a Docker container, one would notice that all supervisord log messages from prior runs of that container would be logged to syslog again. This cluttered the log and made it difficult to follow.
I found that the rsyslog imfile module's statefile would not consistently get updated. According to the documentation, by default, a new state file is only written when the monitored file is being closed (end of rsyslogd execution). It appears that this was not always happening (possibly a race condition between rsyslog exiting and container shutdown?). This behavior can be modified to update the state file after every
nnumber of lines forwarded to the syslog by setting the value of$InputFilePersistStateInterval. Note that setting this to a small value can have an impact on performance. However, supervisord logs are very low-volume (we normally only get 10-20 messages when the container starts and stops--none during normal execution), so I decided to set the value to1in order to ensure the state file is updated after every supervisord message, thus ensuring that we will never see old messages appear in the syslog.